Supervised learning on graphs of spatio-temporal similarity in satellite image sequences

High resolution satellite image sequences are multidimensional signals composed of spatio-temporal patterns associated to numerous and various phenomena. Bayesian methods have been previously proposed in (Heas and Datcu, 2005) to code the information contained in satellite image sequences in a graph representation using Bayesian methods. Based on such a representation, this paper further presents a supervised learning methodology of semantics associated to spatio-temporal patterns occurring in satellite image sequences. It enables the recognition and the probabilistic retrieval of similar events. Indeed, graphs are attached to statistical models for spatio-temporal processes, which at their turn describe physical changes in the observed scene. Therefore, we adjust a parametric model evaluating similarity types between graph patterns in order to represent user-specific semantics attached to spatio-temporal phenomena. The learning step is performed by the incremental definition of similarity types via user-provided spatio-temporal pattern examples attached to positive or/and negative semantics. From these examples, probabilities are inferred using a Bayesian network and a Dirichlet model. This enables to links user interest to a specific similarity model between graph patterns. According to the current state of learning, semantic posterior probabilities are updated for all possible graph patterns so that similar spatio-temporal phenomena can be recognized and retrieved from the image sequence. Few experiments performed on a multi-spectral SPOT image sequence illustrate the proposed spatio-temporal recognition method

Further results on dissimilarity spaces for hyperspectral images RF-CBIR

Content-Based Image Retrieval (CBIR) systems are powerful search tools in image databases that have been little applied to hyperspectral images. Relevance feedback (RF) is an iterative process that uses machine learning techniques and user's feedback to improve the CBIR systems performance. We pursued to expand previous research in hyperspectral CBIR systems built on dissimilarity functions defined either on spectral and spatial features extracted by spectral unmixing techniques, or on dictionaries extracted by dictionary-based compressors. These dissimilarity functions were not suitable for direct application in common machine learning techniques. We propose to use a RF general approach based on dissimilarity spaces which is more appropriate for the application of machine learning algorithms to the hyperspectral RF-CBIR. We validate the proposed RF method for hyperspectral CBIR systems over a real hyperspectral dataset.Comment: In Pattern Recognition Letters (2013

Sentinel 1 and 2: Searching for Physical Image Content

ESA’s upcoming Sentinel-1 and Sentinel-2 missions open new perspectives for the application-oriented use of SAR and/or optical multispectral images. We expect short and regular revisit times as well as easily available and well documented products with attractive features such as cross-polarized SAR images and optical images delivered, for instance, as spectral reflectance data. Thus, users do not have to live any longer with simple digital units or detector counts; instead, the data provided as Sentinel products can be understood as samples of calibrated and validated physical quantities. As a consequence, users can concentrate immediately on the physics and quantitative details of the observable phenomena. This also affects content-based image retrieval, where a user searches for images containing phenomena being similar to given examples. While retrieval systems based on visible image data can only exploit characteristic shapes or patterns, the use of Sentinel data will address the determination of real physical relationships. In particular, this allows a physics-based analysis of image time series data, where one analyzes spatio-temporal phenomena. This physics-based approach will allow us to employ content-based image retrieval as an attractive tool for the analysis of SAR and optical images

Expanding the Algorithmic Information Theory Frame for Applications to Earth Observation

Recent years have witnessed an increased interest towards compression-based methods and their applications to remote sensing, as these have a data-driven and parameter-free approach and can be thus succesfully employed in several applications, especially in image information mining. This paper expands the algorithmic information theory frame, on which these methods are based. On the one hand, algorithms originally defined in the pattern matching domain are reformulated, allowing a better understanding of the available compression-based tools for remote sensing applications. On the other hand, the use of existing compression algorithms is proposed to store satellite images with added semantic value

CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing

We address the problem of cross-modal information retrieval in the domain of remote sensing. In particular, we are interested in two application scenarios: i) cross-modal retrieval between panchromatic (PAN) and multi-spectral imagery, and ii) multi-label image retrieval between very high resolution (VHR) images and speech based label annotations. Notice that these multi-modal retrieval scenarios are more challenging than the traditional uni-modal retrieval approaches given the inherent differences in distributions between the modalities. However, with the growing availability of multi-source remote sensing data and the scarcity of enough semantic annotations, the task of multi-modal retrieval has recently become extremely important. In this regard, we propose a novel deep neural network based architecture which is considered to learn a discriminative shared feature space for all the input modalities, suitable for semantically coherent information retrieval. Extensive experiments are carried out on the benchmark large-scale PAN - multi-spectral DSRSID dataset and the multi-label UC-Merced dataset. Together with the Merced dataset, we generate a corpus of speech signals corresponding to the labels. Superior performance with respect to the current state-of-the-art is observed in all the cases

Accelerated Probabilistic Learning Concept for Mining Heterogeneous Earth Observation Images

We present an accelerated probabilistic learning concept and its prototype implementation for mining heterogeneous Earth observation images, e.g., multispectral images, synthetic aperture radar (SAR) images, image time series, or geographical information systems (GIS) maps. The system prototype combines, at pixel level, the unsupervised clustering results of different features, extracted from heterogeneous satellite images and geographical information resources, with user-defined semantic annotations in order to calculate the posterior probabilities that allow the final probabilistic searches. The system is able to learn different semantic labels based on a newly developed Bayesian networks algorithm and allows different probabilistic retrieval methods of all semantically related images with only a few user interactions. The new algorithm reduces the computational cost, overperforming existing conventional systems, under certain conditions, by several orders of magnitude. The achieved speed-up allows the introduction of new feature models improving the learning capabilities of knowledge-driven image information mining systems and opening them to Big Data environment

Automatic Change Analysis in Satellite Images Using Binary Descriptors and Lloyd–Max Quantization

In this letter, we present a novel technique for unsupervised change analysis that leads to a method of ranking the changes that occur between two satellite images acquired at different moments of time. The proposed change analysis is based on binary descriptors and uses the Hamming distance as a similarity metric. In order to render a completely unsupervised solution, the obtained distances are further classified using vector quantization methods (i.e., Lloyd's algorithm for optimal quantization). The ultimate goal in the change analysis chain is to build change intensity maps that provide an overview of the severeness of changes in the area under analysis. In addition, the proposed analysis technique can be easily adapted for change detection by selecting only two levels for quantization. This discriminative method (i.e., between changed/unchanged zones) is compared with other previously developed techniques that use principal component analysis or Bayes theory as starting points for their analysis. The experiments are carried on Landsat images at a 30-m spatial resolution, covering an area of approximately 59×51 km2 over the surroundings of Bucharest, Romania, and containing multispectral information

A fast compression-based similarity measure with applications to content-based image retrieval

Compression-based similarity measures are effectively employed in applications on diverse data types with a basically parameter-free approach. Nevertheless, there are problems in applying these techniques to medium-to-large datasets which have been seldom addressed. This paper proposes a similarity measure based on compression with dictionaries, the Fast Compression Distance (FCD), which reduces the complexity of these methods, without degradations in performance. On its basis a content-based color image retrieval system is defined, which can be compared to state-of-the-art methods based on invariant color features. Through the FCD a better understanding of compression-based techniques is achieved, by performing experiments on datasets which are larger than the ones analyzed so far in literature

FrFT-Based Scene Classification of Phase-Gradient InSAR Images and Effective Baseline Dependence

In the literature, scene recognition from interferometric synthetic aperture radar (InSAR) images has been mainly focused on the joint use of the backscatter intensity and the coherence between interferometric image pairs. However, the terrain height information residing in the interferometric phase requires further exploration for classification purposes. In this letter, taking the interferometric phase information into account together with the backscatter intensity, the whole complex- valued InSAR image is exploited for feature extraction. In addition, a new complex-valued phase-gradient InSAR (PGInSAR) image is defined. A fractional-Fourier-transform-based feature ex traction, which was proposed for the classification of single-look complex (SLC) SAR images, is adopted for InSAR and PGInSAR images. For patch-based classification, an image database is generated from bistatic pairs acquired from the same terrain with three different effective baselines. The supervised κ-nearest neighbor classification results show that InSAR outperforms SLC by 15%, whereas PGInSAR introduces further 10% improvement over InSAR or a total improvement of 27% over SLC. Moreover, PGInSAR is found to be more robust to effective baseline changes than InSAR, which makes PGInSAR a better candidate for feature extraction

Earth‐Observation Data Access: A Knowledge Discovery Concept for Payload Ground Segments

In recent years the ability to store large quantities of Earth Observation (EO) satellite images has greatly surpassed the ability to access and meaningfully extract information from it. The state-of-the-art of operational systems for Remote Sensing data access (in particular for images) allows queries by geographical location, time of acquisition or type of sensor. Nevertheless, this information is often less relevant than the content of the scene (e.g. specific scattering properties, structures, objects, etc.). Moreover, the continuous increase in the size of the archives and in the variety and complexity of EO sensors require new methodologies and tools - based on a shared knowledge - for information mining and management, in support of emerging applications (e.g.: change detection, global monitoring, disaster and risk management, image time series, etc.). In addition, the current Payload Ground Segments (PGS) are mainly designed for Long Term Data Preservation (LTDP), in this article we propose an alternative solution for enhancing the access to the data content. Our solution presents a knowledge discovery concept, whose intention is to implement a communication channel between the PGS (EO data sources) and the end-user who receives the content of the data sources coded in an understandable format associated with semantics and ready for the exploitation. The first implemented concepts were presented in Knowledge driven content based Image Information Mining (KIM) and Geospatial Information Retrieval and Indexing (GeoIRIS) system as examples of data mining systems. Our new concept is developed in a modular system composed of the following components 1) the data model generation implementing methods for extracting relevant descriptors (low-level features) of the sources (EO images), analyzing their metadata in order to complement the information, and combining with vector data sources coming from Geographical Information Systems. 2) A database management system, where the database structure supports the knowledge management, feature computation, and visualization tools because of the modules for analysis, indexing, training and retrieval are resolved into the database. 3) Data mining and knowledge discovery tools allowing the end-user to perform advanced queries and to assign semantic annotations to the image content. The low-level features are complemented with semantic annotations giving meaning to the image information. The semantic description is based on semi-supervised learning methods for spatio-temporal and contextual pattern discovery. 4) Scene understanding counting on annotation tools for helping the user to create scenarios using EO images as for example change detection analysis, etc. 5) Visual data mining providing Human-Machine Interfaces for navigating and browsing the archive using 2D or 3D representation. The visualization techniques perform an interactive loop in order to optimize the visual interaction with huge volumes of data of heterogeneous nature and the end-user